Statistical Foundations for the Validation of Computer Models

Confidence in computational predictions is enhanced if the potential ‘error’ in these predictions (the difference between the prediction and nature’s outcome in the situation being simulated) can be credibly bounded. The “model-validation” process by which experimental or field results are compared to computational predictions to produce this confidence provides the raw material for characterizing a computational model’s predictive capability in terms of such error limits. In general, the goal is to evaluate predictive capability, first for predictions in the region of experimentation, then, if possible, for predictions in untested regions of applications. This whole process is fundamentally statistical because it requires the acquisition and careful analysis of appropriate data. We establish a statistical model for characterizing predictive-capability and discuss various experimental design and statistical data analysis issues and approaches for resolving them Analyses based on both ‘frequentist’ and Bayesian statistical paradigms are discussed in general in this paper and illustrated in accompanying papers presented at this workshop. The work of the first author was supported by Sandia National Laboratories and the United States Department of Energy under Contract DE-AC04-97AL85000. Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockhed Martin Company, for the United States Department of Energy. **The work of the second author was supported by General Motors and the National Science Foundation, Grant DMS-0103265.